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Transparency and Interaction in 
System Dynamics Based Learning Environments 



Abstract 



The aim of this paper is to describe issues related to interactivity and access to 
underlyling models and representations in system dynamics based learning environments. 
There is some evidence to suggest that system dynamics can be effectively used to 
promote understanding in complex domains (see, for example, Davidsen, 1994, 1996; 
Sterman, 1988, 1997). However, there is insufficient evidence to establish whether or not 
this is in fact the case. Moreover, there has been little effort to address strategic design 
issues pertaining to the effective use of system dynamics in promoting learning in and 
about complex systems. This paper is an attempt to review the relevant conceptual issues 
in promoting learner engagement with appropriate and relevant representations for 
complex domains. A specific approach will be presented and defended, an elaboration 
example will be discussed, and early evaluation results will be reported. 
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Transparency and Interaction in 
System Dynamics Based Learning Environments 



Introduction 

Technology enhanced learning environments are growing in number and prevalence at an 
unprecedented rate in spite of continuing debates in the academic literature with regard to their 
learning effectiveness, their impacts on organizations, and their overall utility to society (see, for 
example. Carter, 1997; Clark, 1994; Kozma, 1994; Lengel, 1997). As a consequence, there are 
many innovative investigations into how people might learn using new technologies, and these 
studies are, in turn, causing much discussion with regard to foundational issues in learning theory 
and instructional design. 

As technologies have become more sophisticated, and as we have learned how to make effective 
uses of technology to promote learning in selected domains and situations, we have become 
more ambitious with regard to our hopes and expectations for technology enhanced learning and 
instruction. More specifically, we are attacking ever more challenging and complex domains as 
targets for improved learning through the use of technology, and such targets include 
environmental policy development 

First, we briefly identify and review relevant learning theories. We then establish a unifying 
perspective in the context of these theories which has clear design and evaluation implications. 
This unifying perspective will then be used to describe a specific framework which provides 
extensive opportunities for learner interactivity and which also capitalizes on learner access to 
explicit models and structures. We conclude with an illustration of this framework and indicate 
other instances close in kind and spirit to ours. 

Theoretical Perspective 

How do people come to acquire complex skills and knowledge? We ask this apparently simple 
question in order to identify relevant assumptions and highlight its complexity. First, much 
learning research proceeds on the following assumptions: 

1 . Learning is a natural, human activity. 

2. The unit of analysis for learning effectiveness is an individual human learner. 

3. Learners are rational. 

An additional assumption prevalent in the educational research community is that instructional 
design is primarily a prescriptive enterprise forming a bridge between descriptive learning 
research and practical development of learning environments (Reigeluth, 1983). 

There is clearly practical value in adopting this research perspective. By varying the instructional 
methods used in certain conditions, one can measure outcomes and study the effects of those 
methods on learning outcomes. If enough data is collected, one then hopes to be able to establish 
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a strong argument for the desirability of a particular method given certain learning conditions, 
thus prescribing how one ought to design instruction to achieve desired outcomes. In this 
perspective, learning is a natural process, and it is theoretically possible to identify how various 
types of learners engage in this process and take those differences into account in the design of 
instruction. Learners are rational in the sense that they are goal-driven, purposeful agents with 
the ability to identify and select reasonably efficient means to achieve goals. Typically, 
conditions are held constant and various interventions (instructional methods instantiated in 
particular learning environments) are investigated. Learning outcomes or effects are then 
measured on individuals. 

Such a research paradigm has in fact produced many useful findings, so it should not be 
discounted. For example, by emphasizing the rationality of learners, designers are able to 
facilitate the identification of learning goals and support activities likely to satisfy the 
achievement of those goals. This line of research emphasizes intentional learning, and generally 
ignores incidental learning, to which it is admittedly much more difficult to apply the conditions- 
methods-outcomes model. Since it is assumed that learners will want to achieve goals efficiently, 
designers can then specify how and when to provide learners with informative feedback on 
progress towards those goals. 

We shall refer to this as the atomistic perspective because it is characterized by an atomistic view 
of learning, both in terms of units of learning (very specific and discrete conditions, methods, 
and outcomes) and in terms of learners (typically focusing evaluation on individual learners, 
even when the setting involves cooperative learning). The atomistic perspective can be 
contrasted with what we shall call the integrated perspective (Spector, 1994, 1995). The 
integrated perspective begins with a view of a person as a member of a society or language 
community. The overall goal of a society or language community typically involves a strong 
survival element, although this is quite often not made explicit. Living consists of working and 
learning, which are viewed as essentially collaborative efforts to achieve commonly held goals. 
From this perspective, individuals might manage to acquire extremely high levels of 
performance at particular tasks while the larger social group consistently falters. This would not 
count as effective learning from an integrated perspective. 

There are definite comparisons to a team perspective when thinking of learning from an 
integrated perspective. A sports team may have several outstanding players, leading the league in 
certain categories, while the team is in last place. This is not a satisfactory situation, especially 
from the team's point of view. One can further imagine a team locked into poor overall 
performance by paying one star player an extremely large salary. This might prevent the team 
from paying higher salaries to others, and it may also foster jealousy and resentment within the 
team. While the star player may draw in large crowds and the team's owners may prosper, the 
team's poor performance may further decline, even when a second star player is added. 

One should be careful not to carry the sports analogy too far, however. Learning organizations 
and societies are not necessarily competitive with other such organizations, as is the case in the 
world of sports. Moreover, membership criteria and reward mechanisms are entirely different. 
Our point here is to suggest that with many situations it is appropriate and useful to take a more 



integrated, holistic view of learning. We believe this is especially important with regard to 
learning in and about complex domains. 

Complex Domains 

We are especially interested in complex domains for a number of reasons. From a research 
perspective, these domains present the most significant challenges, both for designing effective 
learning environments and for determining factors which contribute to learning. From a social 
perspective, these domains present the most significant challenges for the future well-being of 
our species on this planet. We have serious problems to confront if we are to survive, including 
worsening global environmental problems, persisting regional and ethnic conflicts, and wildly 
fluctuating economic conditions. Can we become better prepared to meet these challenges? 

How? 

Complex systems can be depicted as a collection of inter-related items (e.g., stocks and flows in 
system dynamics), and they are characteristics by internal feedback mechanisms, nonlinearities, 
delays, and uncertainties (Sterman, 1988, 1994). These systems typically exhibit dynamic 
behavior, especially in the sense that how they behave has an effect on internal relationships (the 
structure of the system), perhaps strengthening one of the feedback mechanisms (e.g., the 
owners' proceeds due to acquiring a star player reinforces that mode of response to declining 
profits, as opposed, for example to improving overall performance.). This change in internal 
structure in turn has consequences for how the system will behave in the future, for example, the 
team may perform even more poorly as existing players resent the special treatment and salary 
given the star (Davidsen, 1994, 1996). 

Complex systems can be found in abundance at many different levels. The human body can be 
viewed as a complex system. Economics, ecology, epidemiology, project management, and 
training all typically involve complex, dynamic systems. As a species, we have well-documented 
difficulty in dealing effectively (understanding and making robust policies concerning) with such 
systems (see, for example, Ddmer, 1996). There are clearly individual exceptions, persons who 
someone acquire a deep understanding of such systems. Typically, such deep understanding, 
characterized by effective decision making across a wide variety of changing conditions, takes 
years to acquire, and appears not to be easily acquired in spite of concentrated education and 
training efforts (Dreyfus & Dreyfus, 1986). Why have we failed to improve our thinking skills in 
complex domains in spite of such persistent and serious efforts? 

In part, we have not fully understood relevant psychological and sociological factors. In part, we 
have not fully integrated relevant principles about human learning into design praxis. There are 
probably other factors, as well, but we shall hereafter focus on these two, especially the second. 

We should have learned that humans have difficulty in estimating the effects of accumulation 
over time, in predicting the effects of delays, and in calculating nonlinear outcomes (Sterman, 
1994). We should have learned that even well-intentioned persons tend to focus on local 
problems as opposed to whole systems (even when told that a holistic understanding is essential 
for solving particular problems), that people may become cynical and overlook possible 
solutions when their first attempts fail, that people do not communicate effectively in crisis 
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situations, and that inferring underlying system structures from externally viewed system 
behaviors is not an easy task (Ddmer, 1996). Other overlooked lessons can and should be 
identified, but these provide some clear clues as to how we can return to theory and draw 
together relevant principles to guide the construction of learning environments for complex 
domains. 

Theoretical Foundations 

The theoretical foundations for this effort come primarily from socially-situated learning 
perspective, drawing heavily on the views of Bruner (1985), Lave (1988), Piaget (1970), and 
Vygotsky (1978), and from cognitive complexity theory (Spiro, et al. 1987). Within this 
perspective, learning is viewed as an active process of knowledge construction in which learners 
are typically involved with other learners in authentic, problem-solving situations. The need to 
learn created by a realistic problem provides motivation, and interaction with other similarly 
immersed learners provides facilitation. We are favorably inclined to an attitude similar Sfard 
(1998) that emphasizes the need to take into account both an acquisition (static knowledge 
objects with learners acquiring expertise) and a participation metaphor (dynamic knowledge 
objects with learners as active apprentices). Much higher order learning relies on knowledge and 
associated learning activities that might best be supported within the acquisition view. However, 
to progress beyond competent performance and become a proficient expert (Dreyfus & Dreyfus, 
1986), we believe that the participation metaphor with its emphasis on active learner 
participation in socially-situated and problem-oriented settings is crucial. 

Socio-cultural theories of learning and teaching, as inspired by Vygotsky, have addressed the 
instructional use of computers. Some of these include situated learning (Lave, 1988), cognitive 
apprenticeship (Collins et al., 1989), learning by expanding (Engestrdm, 1987), scaffolding, 
cognitive artifacts, distributed cognition,and so on. In short, the theoretical heritage for 
technology enhanced learning is quite rich. Cognitive apprenticeship (Collins et al., 1989) has 
been the foundation for a number of very successful computer-based learning environments. 
These include Sherlock (Lajoie & Lesgold, 1989; Lesgold, Lajoie, Bunzo & Eggan, 1992) which 
made use of the concept of scaffolding where learners are provided with just enough assistance 
to help them construct their own answer to a problem, building on Vygotsky's notion of the zone 
of proximal development. 

What seems most critical of all in technology enhanced learning environments is interactivity. 
This is a much addressed issue in the literature and we will not review that substantial literature 
here. Rather, we shall simply summarize what we believe are the relevant conclusions as 
follows: 

• Doing goes hand-in-hand with learners; learners learn what they do. 

• As learning environments provide more and more opportunities for active learner 
participation, they tend to promote learning; too many opportunities for interaction, 
however, can lead to confusion and disorientation. 

• Cognitive engagement with the subject material is vital. 

• Opportunities for reflection generally promotes learning. 

• Informative feedback is a necessary part of a complete, cognitive engagement. 
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The next section of this paper discusses the implications of these theories and principles for the 
design of learning environments for complex domains. 

Design Implications for Complex Domains 



Sources of Complexity 

It seems clear that the sources of complexity ought to be taken into account when designing 
suppport learning in and about complex systems. For example, if the source for complexity is the 
large number of variables involved, then ways to cluster and chunk these variable for learners 
will be important. On the other hand, if the source for complexity involves non-linear 
relationships of variables, then helping learners identify those non-linearities and associated 
effects is quite important. Likewise, when complexity is a consequence of delays in a system, 
then it is important to help learners identify those delays and their effects. 

Some complexity may be the result of the inherent fuzziness of variables and relationships. Often 
associated with such uncertainties are the effects of human perceptions about a complex system. 
Again, it seems important to help learners identify those kinds of complexity and to determine 
their consequences. One important factor with regard to effects of human perception concerns 
the ability to determine things within the control of actors and agents within the system and those 
things beyond their control. Indeed, many decision makers mistakenly believe some things are 
beyond their control (Ddmer, 1996). 

Aids to understanding 

Learner support should be responsive to the source of complexity. It should also be responsive to 
the needs of the learner. Allowing some learner control with regard to how much and what kind 
of help is provided is known to enhance learning in many situations (Collins, 1991). Moreover, 
providing less and less explicit support for learning is a powerful technique to enhance learning. 
Equally important to improving learning support is to offer multiple representations of various 
complex systems at different levels of aggregation (Sprio et al., 1987). 

Overall outcomes of the intelligent tutoring research has led some to conclude that providing 
dynamic student modeling at a deep level and using that to determine what learning supports and 
materials to present to students remains a generally unresolved computational problem (Pea, 
1993; Spector, 1994; Termyson, 1993). A significant portion of the burden for determining how 
best to support learners is best left to learners. The various illustrations below demonstrate a 
variety of techniques for accomplishing this. 

Transparency 

Perhaps the single most important implication of the analysis of complex systems for the design 
of instruction concerns the relevance of underlying causal models for learning. Many previous 
instructional systems have been black box systems which kept the model hidden from learners. A 



transparent approach provides learners with access to both the causal representation (e.g., a 
causal loop diagram) and the structural representation (e.g., a stock and flow diagram) of a 
complex system. This is entirely in keeping with principles of cognitive flexibility theory (Spiro 
et al., 1987). However, it introduces a new problem. These representations can become 
incredibly complex so the learner should be provided with appropriate chunks and provided 
opportunities to masters those chunks (Davidsen 1994a, 1994b, 1996). Finally, it is important to 
insure that the learner has a clear picture of the overall system before proceeding to individual 
chunks or sectors (Spector & Davidsen, 1996, 1997a, 1997b). 

An Elaborated Example 

This example describes an interactive learning environment for practicing resource allocation in 
large scale instructional development projects. This learning environment contains a system 
djmamics based simulation model, which describes the structure and dynamics of large scale 
instructional development projects. The purpose of the learning environment is to develop users’ 
appreciation and understanding of the many dynamic factors involved in project planning and 
resource allocation, especially those pertaining to how project teams are comprised. Such a 
learning environment allows managers and project participants to gain practice and experience — 
an important factor for running projects successfully. It allows managers to compress time and 
learn from simulated situations by reflecting on the outcomes of decisions. 

Instructional Systems Development (ISD) 

Instructional systems development (ISD) is a reasonably well-structured and well-established 
process for developing educational and training systems and environments. Instruction often 
involves technology-based materials, often referred to as courseware. These systems involve 
significant and expensive software development and typically represent a level of complexity not 
encountered in more typical business-oriented software development projects. Large-scale 
courseware development projects often run behind schedule and over or beyond allocated 
budgets. They frequently use a lot of resources in terms of manpower, especially expertise that is 
already in scarce supply. There is clearly a delay associated with tasks having to be reworked, 
and failure to anticipate such a delay typically causes unexpected schedule overruns. There is 
also a relationship between how many tasks need to be reworked and the composition of the 
workforce (percent of experts and novices working on tasks). Experts typically provide better 
initial estimates of how large a project is and typically create fewer tasks requiring rework. 
Experts also cost more to keep on a project and are often in high demand and short supply. Such 
complex and dynamic factors as these make it difficult to determine optimal resource allocation 
for projects. How, then, can we promote the acquisition of these skills and knowledge? 

We have chosen to use ISD^ (Tennyson, 1993) because it is consistent with system dynamics in 
that it explicitly recognizes internal feedback relationships and represents the system as dynamic 
as opposed to previous more static ISD models. Further, we focus on the allocation of expertise 
in the early stages of a large project, as things that go wrong here are more difficult to correct 
later. 

The Purpose of the Learning Environment 
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The overall purpose of this learning environment is to develop an appreciation and understanding 
of the many dynamic factors involved in project planning and resource allocation, especially 
those pertaining to how project teams are comprised. The more specific goal is to help learners 
develop skills in formulating robust policies for distributed limited expertise across several large 
projects so as to keep the various projects on schedule and within budget. The target audience of 
this learning environment includes managers and project members in large scale ISD Projects. 
The target audience also includes people interested in resource allocation and project 
management in general, especially with regard to problems associated with scare expertise and 
designing effective project teams comprised of both novices and experts. Prerequisite knowledge 
for learners includes familiarity with the basics concepts associated with project management 
(e.g., critical path analysis, project scheduling, etc.), system dynamics and system thinking (e.g., 
causal loop diagrams, internal feedback structures, etc.), and with ISD (e.g., requirements 
analysis, instructional design specification, etc.). 

This learning environment has evolved from an earlier project to provide content specialists with 
a simulation authoring support tool for modeling continuous process systems, and it has been 
refined by a number of graduate students in the System Dynamics Masters Program at the 
University of Bergen (Sioutine et al. 1998; Spector, 1998; Spector & Davidsen, 1996). 

The Simulation Model 

We have been investigating resource allocation problems that arise in the context of ISD project 
management have been investigated using system dynamics theory (Forrester, 1961). The result 
of our research has been a system dynamics based model of ISD project development and 
management (Sioutine et. al., 1998). This model has been incorporated into a learning 
environment that is presented below. Note that the model is easily adjustable to institutional 
practices and should be refined and then validated to reflect what actually occurs within a 
particular organization. However, as with many models, there are key leverage points which 
dominate the system and generalize across a reasonably wide variety of organizational variations 
and local practices. Further, since we are focusing only on large scale instructional development, 
the assumption is that an organization engaged in such activities will have identifiable and 
established procedures and practices, since there will be strong interest in scalability, reuse, 
productivity, and related issues. 

The model depicts the analysis and design phases, as these early stages are often the most crucial 
for a project in terms of ending up within time and budget constraints. Specifically, the attention 
is on the most common ways in which activities in these two phases are interrelated. These 
interrelated activities appear critical to resource allocation, planning, and to getting a project 
started on a productive path. Project phases can be generally described in terms of activities or 
work carried out by various project personnel. These activities are often described as tasks, 
where a task is considered a discrete piece of work or a specific activity with an associated work 
effort necessary for completion. Tasks, then, are identifiable and describable, having a more or 
less well-defined scope, usually measured in terms of person-days from the point of view of 
resource allocation. 
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The underlying model consists of four basic sectors that describe key aspects of the structure of 
the project: (1) human resource management; (2) control; (3) plans, and (4) development. A large 
part of the model is based on the work of Abdel-Hamid and Madnick (1990) applied to the 
modeling of the software project development. Figure 1 depicts the ISD model’s major sectors 
with the main interrelationships between each of the sectors indicated by arrows. 




Figure 1. Sectors and interrelationships in an ISD model. 

Analysis and design tasks can be described as passing through two stages: Identification and 
Processing. Tasks can be completed correctly (successfully) or incorrectly (unsuccessfully). 
Contrary to Abdel-Hamid and Madnick (1990), we have modeled the identification process 
explicitly, because identifying items is highly correlated with level of expertise, which we 
hypothesize is a central component of this system. Figure 1 represents a low level of 
transparency, as it does help develop deep insight into system behavior, but it does support 
understanding that the system is a whole with interrelated parts, and this understanding is a 
necessary building block in developing sensitivity to system behavior. 

The diagram portrayed in Figure 2 represents the flow of tasks inside and between the analysis 
and design phases. Failed tasks eventually must be recognized and reworked. The tasks to be 
reworked can circulate inside a phase as well as between phases. The failure rate depends on the 
experience of the team, workforce composition and the general level of project know-how. The 
failure of a task means a failure of a discrete piece of work or activity, and this implies that it 
must be completed again, possibly effecting the completion or rework of other associated tasks. 
This represents a significant difference between software project modeling (where rework is 
described by errors in software code, and rework is dependent on the density of these errors) and 
instructional project modeling (where rework is described by failed tasks, and rework is 
dependent on the density of other tasks associated with the failed task). The success of rework in 
this formulation is a nonlinear function of the iteration process of tasks and project progress. 
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Figure 2. Interrelations in the analysis and design stages. 

The sector that models human resource management consists of two sub-sectors: (1) the 
allocation of people to tasks and phases; and, (2) the hiring and transfer of people (Sioutine, 
1998). The first sub-sector simulates the allocation (or release) of people on (from) different 
types of job based on the availability of workforce and resource requirements in different phases. 
The second sub-sector simulates the necessary hiring of people or transfer of people out of the 
project. The decisions for hiring or transfers are based on the workforce sought in order to 
complete the project on time. That information comes from the control and plarming sectors. The 
workforce is differentiated into two types: experts and novices. As a consequence of the different 
types of workforce, the productivity of an average person in the project team will be lower than 
the potential productivity of the team (Abdel-Hamid & Madnick, 1989; Steiner, 1972). 

The structure of the model portrays how complex the project management problem is. Therefore, 
the aim of the learning environment based on this model is to show just enough complexity to 
demonstrate to learners the key relationships between the structure and the behavior of the 
system, enabling them to better constitute teams within the early and critical phases of a large 
project. 

The Structure of the Learning Environment 

The “On Time, Within Budget” Learning Environment is a collaborative, simulation-based 
learning environment. It has an underlying model of key project management problems, 
specifically aimed at representing the analysis and design phases of large instructional 
development projects. It has been designed to be played in a local area network with three 
computers (one for each player) and a server. It is possible to play as a team, each team playing 
one of the the roles designed in the game, or one person can play one of the roles on a single 
computer with the other roles being simulated by the computer. Because small groups or teams 
are encouraged, the game is intended as a collaborative learning environment, even though it is 
possible for individuals to learn in isolation from other learners. 
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This learning environment situates players in a simulated replica of a real project. It is 
interactive, because players are required to intervene, design strategies, implement strategies, 
observe results. Players also receive feedback regarding their performance, and they have the 
chance to offer comments at several stages within an activity. The overall design approach used 
in this environment is based on cognitive apprenticeship (Collins, 1991), on socially situated 
learning (Lave, 1988), and on graduated complexity as the primary elaboration sequence 
(Reigeluth, 1983). The notion of graduated complexity in the environment is implemented in 
such a way that users can explore a problem on several levels of complexity, getting more 
sophisticated information at each of the subsequent levels. The whole problem of management is 
separated into several sub-structures: human resource allocation, time planning, process 
development information, personnel information and progress perceptions. Each of these 
structures are represented in the form of an information or decision desk, whereupon the learner 
can choose to get more infomiation regarding that substructure. 

This environment has been designed in such a way that players are first presented with a simple 
representation of information. For example the representation of tasks is symbolised as a kind of 
pile (e.g., pile of papers). The player will normally be given initial information and can then 
optionally explore the problem on three additional levels of complexity, suggested by the 
environment and appropriate to the specific context. Each level provides learners with more 
detailed information on key structural relationships. The general scheme is that at the first level 
learners get information concerning what is being influenced by their decision(s), or what is 
influencing that particular decision point. At the second level, participants get causal and/or 
structural relationships, usually provided with a pop-up causal loop diagram. 




12 



Roles in the Project 




Figure 3. Roles, responsibilities and interactions in the project. 



Finally, at the third level learners will get a simplified structure in terms of a stock and flow 
diagram. At each level learners have the possibility to obtain information about the behavior of 
the system in the form of graphs. This provides learners with a fundamental representation of 
keystructural-behavioral interrelationships in the system. During interactions with the simulation 
environment, learners are asked to record their rational for decisions on the decision record 
sheets. They are asked to record each decision at each particular point and also asked to answer 
this question: “Why did you make this a decision?” 

Domer's (1996) research indicates that providing opportunities for reflection contributes to 
improved learning. As a consequence, learners have to provide at least one short sentence 
indicating why a particular decision was made. This enforced pause for reflection also provides a 
key opportunity for small group interaction, consistent with the collaborative nature of the 
learning environment. While this pause may seem like an additional burden for the learners, it in 
fact provides them with essential opportunities for collaboration and reflection, and avoids the 
impression that theyare just manipulating parameters. The expectation is that learners will 
eventually gain an appreciation for and understanding of the complexities of the system. 

Learners can also record alternative decisions that could have been made at some point and some 
comments on why that decision was not accepted. 

Description of Roles 

In this learning environment participants are play the role of one of the leaders of a multiple 
phase project for large-scale instructional development. The phases of the project involved are 
the early project planning phases of analysis and design. The task for the players will be to 
allocate resources, especially limited expertise, so as to keep the project on time and within 
budget. They will have an opportunity to practice various resource allocation policies in an 
attempt to find a policy, which avoids catastrophic results (late and costly projects). There are 
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three basic roles in the project game to be played: Project Manager, Analysis Task Leader, and 
Design Task Leader. 

The relationships and responsibilities of each of them are portrayed in Figure 3. 

Project Manager . The project manger is the one who controls the pool of workforce in the 
project, does the time planning and sets the completion date for the project. This person is 
responsible for establishing the initial team and its composition in terms of workforce mix, and is 
also responsible for hiring additional people or transfering people from the workforce pool. The 
project manager has to foresee future needs so that there will be enough workforce to complete 
the development. On the other hand, if persons are staying in the pool of available workforce but 
not assigned to any tasks, these persons are standing by, waiting, and are paid for being in the 
project, driving project costs up without associated improvements in productivity. The decisions 
to be taken by the project manager are: 

• how many persons to hire or transfer during the coming decision period; 

• what mix of people to have in terms of percentage of experts and novices; and, 

• set the deadline for the project and the milestones for the phases. 

The allocation policy that participants are currently provided is the simplest one (SAY WHAT 
THAT POLICY IS ! ! !). While other types of policies are implemented in the model (e.g., top 
priority goes to the rework of tasks), the principles of cognitive apprenticeship and graduated 
complexity suggest that learners should start with a simple allocation policy first. 

Analysis Task Leader . The analysis task leader is the leader of the analysis phase of the project. 
This person (or group) is responsible for completion of analysis by allocating resources to 
support analysis and associated rework. This leader and the task leader of the design phase are 
going to use and compete for the workforce resources from the common pool of persons 
available. The leader must eventually learn to keep in mind that it is possible to reallocate 
resources for example from analysis to analysis rework only by releasing workers from one sub- 
phase and making them available in the persons available pool. It is then up to the project 
manager to decide whose requests to satisfy first. The project manager makes this cedision so 
that people will be allocated (assigned new job) and sent to one of the phases or sub-phases 
according to the prioritisation that the manager has. Separating these decisions and roles forces 
learners to develop reasonably sophisticated justifications for resource allocation and assignment 
of scarce expertise to specific tasks and phases. 

Design Task Leader . The design tasks leader is the leader of the design phase of the project. This 
person is responsible for completing the design phase by allocating resources to design and 
associated design rework. This leader is in the same kind of situation as the leader of analysis. 
The leaders of the two phases have to collaborate with each other in other to use the resources 
most effectively. Since there is an interaction and iteration of tasks between the phases, at some 
points it may be necessary to do more of the design in order to discover flaws of analysis, so that 
both phases can proceed further. Therefore, at such moments itthe design phase. This is the 
challenge that participants are facing in this learning environment. The decisions to be taken by 
the analysis and design tasks leaders include the following: 
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• How many persons to request for the coming decision period for processing regular 
analysis or design tasks. 

• How many persons to request for the coming decision period for processing rework 
analysis or design tasks. 

• How many persons can be released from processing of regular analysis or design 
tasks during the coming decision period. 

• How many persons can be released from rework of analysis or design tasks during the 
coming decision period. 

Learning in This Environment 

At the very beginning of the learning experience with the simulator, learners are provided with 
information about their roles and responsibilities, and the requirements of their positions. The 
players are asked to keep the project on time while minimizing costs. In reality, the situation is 
more complicated since quality is a highly critical factor and would also have to be taken into 
consideration. Consistent with the notion of graduated complexity, however, this environment 
emphasizes minimizing costs while still delivering an acceptable end product on schedule. A 
more sophisticated goal could be to deliver the highest quality product without significant cost 
and time overruns, but that is not the learning goal here. 

Referring to the diagram of the project structure in Figure 1, the management of a project 
involves several dynamically interrelated structures. Without looking much deeper it is already 
obvious that the resource allocation situation is complicated, and as we take a deeper look into 
project dynamics, it will become even more complicated. Appreciating and understanding this 
complexity is one of the learning objectives of this simulator. 

To complete the project the players have to complete both phases. At the beginning, there is 
some estimate of the project size in terms of analysis and design tasks. This estimate may be 
different from the real size at the beginning, and, therefore, will need to be adjusted as simulation 
proceeds through the development of the project. The priority for allocation of people initially is 
that people are first allocated to Analysis, then to Analysis Rework, then to Design, and finally to 
Design Rework. Therefore, if there are multiple requests for allocations to different sub-phases 
(e.g., regular Analysis or Analysis Rework, or regular Design and Design Rework), first priority 
will be to satisfy requests for allocation to regular analysis, then to analysis rework, then to 
regular design, and finally to design rework. Later allocations to subsequent sub-phases are made 
depending on the availability of workforce in the pool of Persons Available. 
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Figure 4. An example of the Main Office screen for the Design Task Leader. 

The project learning environment was designed to be installed in a computer network with three 
local computers and a server. This is an asymmetric game, with different persons/teams playing 
different roles and making different decisions The common goal is to complete the project on 
time and within budget. Each player/role has its own interface representing the corresponding 
role to be played. However, each of the interfaces are similar in the representation of the 
information and metaphors that represent certain concepts. 

The way the project learning environment proceeds is that players are initially provided with 
information about the project environment, its characteristics, the aims and prerequisites for “On 
Time, Within Budget,” and how to navigate, get information, and make decisions.. Thereafter, 
players can optionally 

• get help on navigation; 

• ask for the project description; 

• get information on the project structure; or, 

• ask for a role description. 

The players then enter the project environment based on the role that has been selected or 
determined by the facilitator. Players are allowed two choices: 

• Play With Computer; or, 

• Play Together With Other Players. 

The first choice allows players to run the project in the role that has been described in the 
introduction section in “collaboration” with other players simulated by the computer. That is, the 
other two roles will be simulated according to the policy and decision structure built into the 
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underlying model for the corresponding roles. The second choice involves all of the players as 
real persons/groups playing each role on different computers. 

For the purpose of practice and familiarization, it is suggested that players first play with the 
computer simulating the other roles, as the need to collaborate with those playing the other roles 
is an additional level of complexity. After learners have gained familiarity with the environment, 
they are prepared to initiate a collaborative game. It is also recommended that all learners 
eventually play each of the different roles in order to get an appreciation of the problems 
associated with that role and the possible misperceptions of others involved in the decision 
process. 

The metaphor underlying “On Time, Within Budget” consists of the idea that users are in a 
virtual office. From that place they can get to the areas of interest through doors leading to a 
corresponding place. Once players start the project they are placed into their MAIN OFFICES. 
This place connects them to all of the information sources that they have in the project learning 
environment. The layout of the MAIN OFFICE is exhibited in Figure 4. From the “Main Office” 
players can chose the following areas, by following in to the corresponding door: 

• DECISION DESK: the decisions and assumptions can be tested and then 
implemented. 

• SCHEDULE DESK for the project manager and CALENDAR for the task leaders. At 
SCHEDULE DESK, the project manager sets the completion date and analysis 
milestone. At CALENDAR, the two phase leaders receive the information about the 
completion date and time remaining. 

• PERSONEL INFORMATION: the information about the workforce level, 
productivity and time utilization. 

• PROGRESS REPORT: provides the information about project accomplishment and 
resources used. 

• TASKS/JOB INFORMATION: provides the information about work load on 
different stages of the project’s phases (identification, processing, rework recognised, 
rework, accomplishment) 

The Two Modes of Interaction: Decision and Strategic 

During the interaction with the learning environment players will have the possibility to run the 
simulation in two modes: strategic and decision. The strategic mode (or Try Decisions) allows 
players to test decisions, before accepting them into the project. This allows plays to try out 
diverse strategies for each of the scenarios, therefore giving players the chance to look into the 
future and evaluate the possible impacts of their decisions. These trial decisions are not 
submitted to other players/teams but are only tested locally and discussed by a particular team as 
a possible strategy to pursue. For this reason, players have to make assumptions about the 
decisions of the other teams. This possibility is provided at the “DECISIONS” desk by the 
“TEST ASSUMPTIONS” option. This brings up the “ASSUMPTIONS” panel where players are 
provided with virtual control over the decision variables of the two other players. After players 
have tested enough strategies and made up their minds, they are ready for switching to the 
Decision Mode by choosing “Revert to Game” option on the game control panel. The players 
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will then be asked if they want to keep the current decisions or not. Finally, when a team is ready 
to make decisions, they chose “Accept Decisions,” which will deliver decisions to the other 
players. As soon as everybody is ready with their decisions, the project will proceed further and 
players will soon get another request for their decisions. 

The players are asked to make their decisions every 10 days, and submit them when they are 
asked for. However, the decision interval can be changed by the facilitator. The project runs until 
its completion or until it falls behind schedule, which means that a team was not able to complete 
the project within the scheduled deadline. As the simulation unfolds the players will eventually 
get feedback on their performance and the state of the project. This messages are designed to 
gain the attention of the players. For example, when the project management team perceives that 
a projects falls behind schedule and requires overtime and rework, a message appears indicating 
that the workforce is about to become exhausted. Suggestions with regard to possible actions are 
available (e.g., review the schedule of the project, or add more people, etc.). 

Findings 

Learning is a feedback process which has several barriers that prevent successful outcomes in 
complex dynamic systems (Ddmer, 1996; Sterman, 1994). These barriers include the complexity 
of the system itself, the future and possibly delayed effects of decisions, and various non-linear 
and somewhat uncertain relationships within the system. The use of simulation-based learning 
environments allows for compressing d 5 mamics that would otherwise take place over long time 
periods, thereby overcoming some traditional barriers to successful learning in this domain. The 
“On Time, Within Budget” Learning Environment presented in this paper provides a laboratory 
tool for learning and practising the management of complex project management skills, 
particularly resource allocation and planning project schedules and milestones. 

A special concern during the design of this learning environment was also given to the 
translation of the external instructional events described by Gagne (1995). These learning events 
are as follows: gaining attention, informing of the objective, stimulating recall of prior 
knowledge, presenting information and content, providing “learning guidance,” eliciting 
performance, providing feedback, assessing performance and enhancing retention and transfer. A 
certain degree of transparency for the learner is achieved by representing structural relationships 
using feedback loop diagrams and structural diagrams. These provide learners with a 
representation of the model structure and allow for better understanding of the structural- 
behavioral relationships existing in the system. (Davidsen 1994). 

The learning environment presented in this paper offers a tool for practicing resource allocation 
in project management. This environments facilitates learning and transfer of the knowledge 
about the existing problems using an underlying system d 5 mamics model and simulation. It 
portrays general ideas and problems of project management. Therefore, it can be used not only 
for instructional projects but for other types of projects as well. The next step in the development 
of the instructional material would be an integration it with the other components of the Bergen 
project, which provide tutorials in the areas of project management, instructional development, 
system d 5 mamics and system thinking. Such an effort would produce a highly engaging virtual 
micro-world for resource allocation in large scale instructional projects. 
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Our investigations into black-box learning environments shows that there is inadequate learning 
support and that learning outcomes are marginal. Moreover, investigations in the effects of 
understanding specific issues associated with complex domains (e.g., recognizing and predicting 
effects of second order delays) indicates that providing learners with access to underlying 
models, along with opportunities to perform both hypothetical strategy formulation and active 
decision making in a realistic setting, can produce learning benefits (Bahaa et ah, ). 

Concluding Remarks 

It is premature to argue that a framework such as the one presented here will produce significant, 
long-lasting, and positive learning effects in and about complex systems. Based on existing data 
collected on our learning environments and from data reported on some of those which adopt 
similar approaches, we believe that there is great promise in designing collaborative teleleaming 
environments from a socially-situated learning perspective with heavy emphasis on collaborative 
learner participation in the creation and modification of knowledge objects and artifacts. We look 
forward to learning about the results and findings of others in this area. 

We have reported some of our findings at meetings of the International System Dynamics 
Society (Sioutine, Davidsen, & Spector, 1998; Spector & Davidsen, 1997b), and we have been 
encouraged to learn that the system dynamics community is coming around to the view that it is 
not generally sufficient to provide only behaviorally oriented feedback about complex systems if 
one wishes to promote deep understanding of such systems. Learners should be encouraged to 
engage in model alteration, model construction, and policy and strategy design in a collaborative 
context if one expects lessons to transfer from the learning environment to real-world settings. In 
addition, we have presented this framework to the Competence-Based Management Conference 
(Spector, 1998), and we are encouraged to learn that this approach appears relevant to the 
teaching of skills and expertise pertaining to management. 
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